Systems and Methods for Interacting with Three-Dimensional Graphical User Interface Elements to Control Computer Operation

ABSTRACT

Disclosed are three-dimensional (“3D”) graphical user interface (“GUI”) elements for improving user interactions with a digital environment or a device by simplifying access to different data, functionality, and operations of the digital environment or the device. A 3D GUI element may include first visual information at a first position and second visual information at a second position within the 3D space represented by the 3D GUI element. In response to first input directed to the first visual information, the 3D GUI or system may perform a first action that is mapped to the first input and the first visual information within the 3D GUI element. In response to second input directed to the second visual information, the 3D GUI or system may perform a second action that is mapped to the second input and the second visual information within the 3D GUI element.

CLAIM OF BENEFIT TO RELATED APPLICATIONS

This application is a continuation of U.S. nonprovisional applicationSer. No. 17/838,393 entitled “Systems and Methods for Interacting withThree-Dimensional Graphical User Interface Elements to Control ComputerOperation”, filed Jun. 13, 2022. The contents of application Ser. No.17/838,393 are hereby incorporated by reference.

BACKGROUND

The two-dimensional (“2D”) nature of icons, emojis, wallpapers, andother graphical user interface (“GUI”) elements restricts theirfunctionality and interactivity. For instance, the amount of informationthat may be presented in two dimensions is less than the amount ofinformation that may be presented in three dimensions. Similarly,interactions with a 2D GUI element may be restricted to the x and yplanes, whereas interactions with a three-dimensional (“3D”) GUI elementmay be performed about the x, y, and z planes.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of three-dimensional (“3D”) icon for anapplication in a closed, minimized, inactive, sleep, and/or unopenedstate in accordance with some embodiments presented herein.

FIG. 2 illustrates an example of dynamically manipulating a 3D icon inresponse to different conditions or events and in accordance with someembodiments presented herein.

FIG. 3 illustrates an example of launching an application using a 3Dicon in accordance with some embodiments presented herein.

FIG. 4 illustrates an example of using different gestures to accessdifferent information from a 3D icon in accordance with some embodimentspresented herein.

FIG. 5 illustrates an example of dynamically manipulating a 3D emojibased on different user input and/or different rules associated with the3D emoji in accordance with some embodiments presented herein.

FIG. 6 illustrates an example of a 3D GUI element for invoking differentfunctionality within an open, active, or frontmost application inaccordance with some embodiments presented herein.

FIG. 7 illustrates an example of a 3D GUI element for controlling filesystem access in accordance with some embodiment presented herein.

FIG. 8 presents a process for presenting and interacting with a 3D GUIelement in accordance with some embodiments presented herein.

FIG. 9 illustrates example components of one or more devices, accordingto one or more embodiments described herein.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements.

Provided are systems and methods for interacting with three-dimensional(“3D”) graphical user interface (“GUI”) elements to control computeroperation. The systems and methods may generate the 3D GUI elements frompoint clouds and/or other 3D file formats, may dynamically manipulatethe 3D GUI elements with different 3D effects based on different sensoroutputs, conditions, events, and/or user input, and/or may control theoperation of a computer or other electronic device based on the 3Dposition at which a 3D GUI element is presented when input is providedrelative to that GUI element.

The systems and methods may therefore include a GUI with one or more 3DGUI elements for simplifying and improving user interactions with adigital environment or a device, and for accessing different data,functionality, and operations of the digital environment or the device.Due to the different computer functionality that is performed inresponse to different 3D positional data associated with an interactedwith 3D GUI element, there are no two-dimensional (“2D”) counterparts oradaptations of 2D GUI elements for implementing the same controls orfunctionality.

The 3D GUI elements may include 3D icons, 3D emojis, 3D wallpapers,and/or other 3D objects that present different visual information and/orare associated with different executable functionality at differentpositions within the 3D space (e.g., different x, y, and z positions)represented by the 3D GUI element. In some embodiments, the 3D icons mayinclude 3D GUI elements for launching an application, receiving dataabout an application, and/or otherwise interacting with an applicationin a closed, minimized, inactive, sleep, and/or unopened state. In someembodiments, the 3D icons may include 3D GUI elements within anapplication that has focus or is in an open or active state. In somesuch embodiments, a single 3D GUI element may be used to invokedifferent functions or operations of the application or to performvarious interactions with the application.

In some embodiments, a 3D GUI element may be rotated and/or interactedwith in various ways to expose the different visual informationcontained in the 3D space of that 3D GUI element. In some embodiments,the same 3D GUI element may be interacted with to invoke differentfunctionality that is associated with the different visual informationwhen that visual information is presented on the 3D GUI element.

The 3D GUI and 3D GUI elements may be implemented on various interactivedevices that have a display. For instance, the 3D GUI and 3D GUIelements may be presented on a smartphone, handheld mobile device,laptop computer, desktop computer, headset, Internet-of-Things (“IoTs”)device, controllers, and/or wearable devices (e.g., smartwatches).

FIG. 1 illustrates an example of 3D icon 100 for an application in aclosed, minimized, inactive, sleep, and/or unopened state in accordancewith some embodiments presented herein. 3D icon 100 may include a 3Dimage that is rendered from x, y, and z positional data andnon-positional data (e.g., visual information defined for each of the x,y, and z positions) contained within a point cloud or other 3D imagefile format. In some embodiments, 3D icon 100 may be presented as avolumetric, cubic, or spherical object. 3D icon 100 may be associatedwith one or more rules that control the rotation or movement of 3D icon100 in response to no inputs, different sensor outputs, detectedactions, detected events, and/or input that are directed to 3D icon 100or the application that is accessed using 3D icon 100.

Different visual information may be defined at different positionswithin the point cloud or 3D image for 3D icon 100. For instance, at adefault position, 3D icon 100 may include a first set of data pointswith positional and non-positional elements that generate a closed trashcan image in the foreground, a second set of data points with positionaland non-positional elements that generate a restore trash can imageabout the left side, a third set of data points with positional andnon-positional elements that generate an empty trash can image about theright side, and a fourth set of data points with positional andnon-positional elements that generate an open folder image in the back.Specifically, the first set of data points may include x, y, and zcoordinates that position the first set of data points towards the frontof the point cloud or 3D icon 100 when rendering 3D icon 100 from thedefault position, the second set of data points may include x, y, and zcoordinates that position the second set of data points about the leftside of the point cloud or 3D icon 100 when rendering 3D icon 100 fromthe default position, the third set of data points may include x, y, andz coordinates that position the third set of data points about the rightside of the point cloud or 3D icon 100 when rendering 3D icon 100 fromthe default position, and the fourth set of data points may include x,y, and z coordinates that position the fourth set of data points aboutthe rear of the point cloud or 3D icon 100 when rendering 3D icon 100from the default position.

In some embodiments, the 3D GUI may rotate 3D icon 100 to present thedifferent visual information in response to sensor output that indicatesa rotation or movement of the device presenting the 3D GUI. Forinstance, the device may include a gyroscope or accelerometer thatmeasures a rightward rotation of the device, and 3D icon 100 may bedefined with a rule that generates a complementary rotation of 3D icon100 based on the rightward rotation of the device. The rightwardrotation of the device may cause movement away from the visualinformation about the front side of 3D icon 100 and movement towards thevisual information about a left side of 3D icon 100. In other words, the3D GUI may render different data points from the point cloud of 3D icon100 at the position of 3D icon 100 within the 3D GUI as the deviceitself rotates. In some embodiments, the device may include a headsetworn by a user, and 3D icon 100 may rotate in response to head movementsor tracked eye movements. In some other embodiments, the device mayinclude sensors that detect or map the position and/or orientation ofthe user (e.g., the user's face or head) relative to the device.Accordingly, in response to detecting movement of the user in a firstdirection, the 3D GUI may rotate 3D icon 100 in the corresponding firstdirection or an opposite second direction depending on the mapping ofthe detected motion to icon controls.

In some embodiments, the 3D GUI may rotate 3D icon 100 in response touser input provided to, over, or for 3D icon 100. For instance, anupwards swipe gesture on 3D icon 100 may cause 3D icon 100 to rotate inthe direction and with a velocity associated with the swipe gesture.Similarly, a detected mouse click while a cursor is placed over 3D icon100 followed by an upward movement of the mouse may cause acorresponding rotation of 3D icon 100 that is determined based on thedirection of the mouse movement as well as the speed or distance themouse is moved. Specifically, rotating 3D icon 100 in response to userinput may include updating a render frustum or camera position withinthe corresponding point cloud or 3D image file based on the directionand magnitude of the input, and presenting or rendering the data pointsof the point cloud or 3D image file from the updated render position.

As shown in FIG. 1 , user input that rotates (at 102) 3D icon 100 to theleft may change the visualization of 3D icon 100 to present the secondset of data points in the foreground. Similarly, user input that rotates(at 104) 3D icon 100 to the right may change the visualization of 3Dicon 100 to present the third set of data points in the foreground.

In some embodiments, the same 3D rendering effects may be applied to a3D wallpaper or image. The 3D wallpaper or image may include a highresolution 3D rendering and/or visualization that is generated from aparticular camera position within the 3D space of a point cloud or other3D image file. The device on which the 3D wallpaper or image isdisplayed may rotate or move. The 3D GUI may apply a rotation ormovement provided based on user input or other interaction to theposition of the camera within the point cloud or other 3D image file,and may render the 3D wallpaper or image from the updated position ofthe camera. Accordingly, the 3D GUI continues rendering the same pointcloud or 3D image albeit from different positions that are determinedfrom different rotations or movements of the device.

Different executable functionality may be associated with the differentsets of data points forming 3D icon 100, and may be invoked when userinput is applied over each set of data points. For instance, the datapoints within each particular set may be encoded with or linked to afunction call, application programming interface (“API”) call, systemcall, executable script, and/or other computer code such that whenparticular input (e.g., a tap, double click, selection, etc.) isdirected over or to any data point of the particular data point set, thecorresponding function, call, or code may be executed.

Accordingly, tapping, double clicking, or otherwise selecting (at 106)the closed trash can image when the first set of data points aredisplayed frontmost in 3D icon 100 may cause a recycle bin applicationor function associated with 3D icon 100 to open and present recentlydeleted files or folders. Tapping, double clicking, or otherwiseselecting (at 108) the restore trash can image when the second set ofdata points are displayed frontmost in 3D icon 100 may cause the recyclebin application to restore the recently deleted files or folders.Tapping, double clicking, or otherwise selecting (at 110) the emptytrash can image when the third set of data points are displayedfrontmost in 3D icon 100 may cause the recycle bin application topermanently delete the recently deleted files or folders. Tapping,double clicking, or otherwise selecting the open folder image when thefourth set of data points are displayed frontmost in 3D icon 100 maycause the 3D GUI to change 3D icon 100 and render a new point cloud thatpresents visual information for one or more of the recently deletedfiles or folders within the recycle bin application. Accordingly,different positions or different sets of data points within the samepoint cloud or 3D image file used to render 3D icon 100 may beassociated with and used to access different functionality.

In some embodiments, the 3D GUI may rotate 3D icon 100 in response to arule that controls the rotation and/or movement of 3D icon 100 whenthere is no sensor output, actions, events, or input being directed to3D icon 100. For instance, the rule may be defined to create anon-stationary icon so that visual information from all sides of 3D icon100 are visible even when the device is not rotated or moved.

FIG. 2 illustrates an example of dynamically manipulating 3D icon 200 inresponse to different conditions or events and in accordance with someembodiments presented herein. 3D GUI 201 may detect (at 202) a conditionor event that is directed to 3D icon 200 or application 203 that isaccessed using 3D icon 200. 3D icon 200 may be generated from renderinga plurality of data points that are defined with positional (e.g., x, y,and z coordinate values) and non-positional elements (e.g., red, blue,green, chrominance, luminance, translucence, and/or other descriptivecharacteristics) in a point cloud or another 3D file format. 3D icon 200may be used to launch social media application 203. Social mediaapplication 203 may receive a new post or notification while running inthe background, via a push notification, periodic updating, and/or otherupdates. Social media application 203 may notify or otherwise message 3DGUI 201 of the new post or notification.

Rather than add a badge to identify the new post or notification as isdone for 2D icons, 3D icon 200 may be associated with different rulesthat specify different 3D rotations, movements, or other manipulationsof 3D icon 200 in response to the different conditions or events. Asshown in FIG. 2 , 3D icon 200 may include a 3D image that includes thefaces or photographs of different friends or contacts within socialmedia application 203. 3D GUI 201 may identify (at 204) the set of datapoints, that are linked to or correspond to the image of the particularfriend, within the point cloud or 3D image from which 3D icon 200 isgenerated, and may render (at 206) the point cloud or 3D image file tocenter on the set of data points. Other data points for images of otherfriends or contacts that fall within the render frustrum may also berendered in the sides or background of 3D icon 200. In some embodiments,3D GUI 201 may gradually adjust the render position within the pointcloud or 3D image file to move from a last render position to theposition and orientation that is centered on and in front of the set ofdata points representing the image of the particular friend. In sodoing, 3D GUI 201 may generate (at 206) a 3D animation effect thatrotates 3D icon 200 to the image of the particular friend and that zoomsin on the image of the particular friend so that the face or photographof the particular friend becomes the focus of 3D icon 200 and consumes alarger portion of 3D icon 200.

In response to new or unread posts from multiple friends, 3D icon 200may rotate back and forth and/or zoom in and out between the faces orphotographs of each friend with a new or unread post. In contrast to ananimated 2D icon that may switch between different images, 3D icon 200is a single 3D image created from a single point cloud or 3D image fileformat, and the associated movements may include moving to andpresenting visual information from different parts within that single 3Dimage or 3D icon 200.

FIG. 3 illustrates an example of launching an application using 3D icon300 in accordance with some embodiments presented herein. 3D icon 300may be used to launch or access a mapping application. 3D icon 300 maybe a 3D image of the Earth or a globe that may be generated from a firstlow resolution point cloud or 3D image file of the Earth or globe.Specifically, 3D icon 300 may be generated from rendering the datapoints of the first low resolution point cloud or 3D image file from afirst distant position.

3D GUI 201 may rotate (at 302 and 304) 3D icon 300 based on positionaldata received from a Global Positioning System (“GPS”) sensor and/orother geolocational services of the device. For instance, 3D GUI 201 mayrotate (at 302) 3D icon 300 to center on a first detected position ofthe device, and may rotate (304) 3D icon 300 to center on a seconddetected position of the device. In some embodiments, 3D icon 300 mayrotate and zoom in to show the particular state or city in which thedevice is located with at the first resolution of the point cloud or 3Dimage file.

3D GUI 201 may detect (at 306) user input that launches the mappingapplication. For instance, user input may include a touch, tap, or othergesture applied to or over 3D icon 300.

Launching the mapping application may include retrieving (at 308) asecond high resolution point cloud or 3D image file for 3D icon 300. Insome embodiments, the second high resolution point cloud or 3D imagefile may include more data points for representing the Earth or globethan the first low resolution point cloud or 3D image file. In someembodiments, 3D GUI 201 may select between different point clouds thatare high resolution scans of different countries, states, or regionsbased on the current position of the device.

Launching the mapping application may further include performing (at310) a first 3D zoom effect in which 3D icon 300 is enlarged and shownfrom a closer vantage point that occupies all or most of the display,and/or performing (at 310) a second 3D zoom effect in which the 3D imagezooms in from the faraway view of the Earth or globe to the exactstreet, block, or address where the device is located. Specifically, 3DGUI 201 may perform (at 310) the zoom effects by enlarging 3D icon 300and rendering different images of the second high resolution point cloudas the render position moves progressively closer inside the second highresolution point cloud to the street location of the device. The 3Deffect created by 3D GUI 201 launching the mapping applicationassociated with 3D icon 300 is not generated by switching betweendifferent 2D image files. Instead, the 3D effect is generated by movingwithin a point cloud to present different visual information containedtherein.

FIG. 4 illustrates an example of using different gestures to accessdifferent information from a 3D icon in accordance with some embodimentspresented herein. As shown in FIG. 4 , 3D icon 400 may include a GUIelement for accessing or opening email application 401.

The 3D GUI may dynamically update 3D icon 400 to layer visualinformation related to different unread emails in the 3D spacerepresented by 3D icon 400. For instance, the 3D GUI may dynamicallymodify (at 402) 3D icon 400 to identify new unread emails that havearrived from different senders, and may layer, stagger, or otherwisearrange the notifications for each email in the 3D space of 3D icon 400.3D icon 400 may be updated to present the sender information and/orsubject line from a most recently received unread email or message.Specifically, as each unread message notification is received by thedevice, 3D GUI, or email application 401, the 3D GUI may modify thepoint cloud or 3D image file used to create 3D icon 400 to include avisualization for that unread message notification. In some embodiments,the 3D GUI may add or define a set of data points at a different planewithin the GUI to provide a visualization for each unread messagenotification. In some such embodiments, 3D icon 400 may include acertain number of planes that limit the number of unread messagenotifications that may be presented or visualized within 3D icon 400.

Different user gestures may then be used to access different unreademails directly from 3D icon 400. For instance, a right swipe gestureapplied to 3D icon 400 may cause 3D icon 400 to rotate and present (at404) the sender information and/or subject line from the next mostrecently received unread email or may invoke right scroll functionalitythat continues to rotate between the notifications for the unreadmessages in 3D icon 400 until the gesture is stopped. In this manner,the user may view visual information about the unread emails withoutopening email application 401.

After halting the right swipe gesture on a particular email message in3D icon 400, a different gesture may be used to open (at 406) emailapplication 401 directly to that particular email message. For instance,an up swipe gesture while the visual information for the particularemail message is displayed centrally within 3D icon 400 may cause the 3DGUI to open email application 401 and invoke the email reply function sothat email application 401 opens directly to a screen in which the usermay enter text for a reply message to the particular email message. Morespecifically, the 3D GUI may determine which set of data points arepresented frontmost or centrally within 3D icon 400 when the up swipegesture is issued, may determine the particular email message that isassociated with or linked to the set of data points, may determine thecommand or function (e.g., reply message function) that is mapped to theup swipe gesture and 3D icon 400, and may perform the command orfunction with the particular email message specified as a variable toopen email application 401 directly to the reply message interface forthe particular email message.

Similarly, a down swipe gesture applied over 3D icon 400 while theparticular email message is centered within 3D icon 400 may invoke (at408) a delete or archive email function of email application 401directly from 3D icon 400 without opening, launching, and/or directlyaccessing email application 401. In other words, the user may delete,archive, or invoke other functions or operations of email application401 on specific emails based on gestures or inputs provided directly to3D icon 400 while email application 401 remains in a closed, minimized,inactive, sleep, and/or unopened state. Once again, the 3D GUI maydetermine which set of data points are presented frontmost or centrallywithin 3D icon 400 when the down swipe gesture is issued, may determinethe particular email message that is associated with or linked to theset of data points, may determine the command or function (e.g., deletemessage function) that is mapped to the down swipe gesture and 3D icon400, and may perform the command or function with the particular emailmessage specified as a variable to delete the particular email message.

Similar functionality and/or operations of other applications may beinvoked for other applications in a closed, minimized, inactive, sleep,and/or unopened state based on gestures or user inputs provided to 3Dicons of those applications, and based on a mapping of the applicationfunctions to different gestures applied to the corresponding 3D icons.For instance, a user may discard or mark social media posts or textmessages as read based on different gestures applied to the 3D icon forthe associated social media application or text messaging applicationwhile the 3D icon is rotated to and/or centered on those social mediaposts or text messages, may delete, edit, or share photographs based ondifferent gestures applied to the 3D icon for a closed, minimized,inactive, sleep, and/or unopened camera or photograph viewingapplication while the 3D icon is rotated to and/or centered on thosephotographs, and/or may receive, send, accept, or decline payments basedon different gestures applied to the 3D icon for a closed, minimized,inactive, sleep, and/or unopened payment application while the 3D iconis rotated to and/or centered on notifications regarding those payments.

In some embodiments, the 3D GUI elements may include 3D emojis. A 3Demoji may include a 3D image that is generated from the positional dataand non-positional data of different points defined within a pointcloud. Rules may be defined for the 3D emoji and/or different datapoints of the 3D emoji to automatically modify the 3D emoji in responseto different detected events, conditions, messages, and/or inputs.

FIG. 5 illustrates an example of dynamically manipulating 3D emoji 500based on different user input and/or different rules associated with the3D emoji in accordance with some embodiments presented herein. 3D emoji500 may be sent in a message exchange between two or more users. Forinstance, 3D emoji 500 may be sent by a first user in response to a textmessage, social media post, shared image, or other content sent by asecond user.

A first rule associated with 3D emoji 500 may define and generate (at502) a first set of rotations or movements to apply to 3D emoji 500 whenpresenting 3D emoji 500 to the second user. The first set of rotationsor movements may include rendering the point cloud or data pointsassociated with 3D emoji 500 from a first set of positions as specifiedby the first rule.

A second rule associated with 3D emoji 500 may define and generate (at504) a second set of rotations, movements, and/or 3D effects to apply to3D emoji 500 based on a response provided by the second user after beingpresented with 3D emoji 500 and the first set of rotations or movements.For instance, the second user may respond with a message that includesthe phrase “LOL”, and the phrase may be a keyword that is tied to a rulethat triggers the second set of rotations, movements, and/or 3D effects.The second set of rotations, movements, and/or 3D effects may includeadjusting the positioning of the point cloud data points to create the3D effect illustrated in FIG. 5 .

In some embodiments, the 3D GUI elements may replace 2D buttons, menus,sliders, and/or other 2D GUI elements within an open, active, orfrontmost application. In some such embodiments, a single 3D GUI elementmay be used to replace and/or invoke the functionality associated withmultiple 2D buttons, menu items, sliders, and/or 2D GUI elements.

FIG. 6 illustrates an example of 3D GUI element 600 for invokingdifferent functionality within an open, active, or frontmost application601 in accordance with some embodiments presented herein. 3D GUI element600 may be a single interactive element that replaces the back, forward,and stop elements of web browser application 601. In some embodiments,3D GUI element 600 may be a single interactive element that replaces thefast forward, rewind, and stop elements of a media player applicationand/or that controls other operations of other applications.

3D GUI element 600 may include a 3D image that is generated fromrendering a single point cloud or other 3D image file from a first 3Dposition. Specifically, 3D GUI element 600 may present first visualinformation when the point cloud is rendered from the first 3D positionas a result of no user input being applied to 3D GUI element 600.

The 3D GUI associated with application 601 may detect first user inputthat is applied to 3D GUI element 600 and that invokes the browser backbutton functionality. For instance, the first user input may include aleft swipe gesture.

In response to the first user input, the 3D GUI may rotate (at 602) 3DGUI element 600 to present second visual information that indicatesinvocation of the browser back functionality. Specifically, the firstuser input may trigger a first rule that causes the 3D GUI to render thepoint cloud from a second position at which the second visualinformation is visible. Additionally, browser application 601 mayretrieve and display the last accessed site in the browser history inresponse to the invocation of the browser back button functionality.

The 3D GUI may detect second user input that is applied to 3D GUIelement 600 and that invokes the browser forward functionality. Forinstance, the second user input may include right swipe gesture.

In response to the second user input, the 3D GUI may rotate (at 604) 3DGUI element 600 to present third visual information that indicatesinvocation of the browser forward functionality. Specifically, thesecond user input may trigger a second rule that causes the 3D GUI torender the point cloud from a third position at which the third visualinformation is visible. Additionally, browser application 601 mayretrieve and display (at 604) next accessed site in the browser historyin response to the invocation of the browser forward buttonfunctionality.

The 3D GUI may detect third user input that is applied to 3D GUI element600 and that invokes the browser halt functionality. For instance, thethird user input may include a touch and hold or tap gesture on 3D GUIelement 600. In response to the third user input, the 3D GUI may cause3D GUI element 600 to rotate back to the first visual information and towobble the first visual information to indicate that the browser haltfunctionality has been invoked.

In some embodiments, the 3D GUI elements may be used to control filesystem access and/or navigation through a directory structure. FIG. 7illustrates an example of 3D GUI element 700 for controlling file systemaccess in accordance with some embodiment presented herein.

As shown in FIG. 7 , the 3D GUI may generate 3D GUI element 700 based ona point cloud or other 3D image file that graphically represents aparticular folder structure. For instance, 3D GUI element 700 mayinclude visual information for different folders or directories as wellas files under a particular parent folder or directory.

First user input may rotate 3D GUI element 700 in order to expose thevisual information for the different contained folders, directories,and/or files by rendering the point cloud or 3D image file fromdifferent render positions. For instance, an up swipe gesture followedby a right or left swipe gesture may be used to access and/or present(at 702 and 704) the visual information for the different folders ordirectories under the particular parent folder or directory. Similarly,a down swipe gesture followed by a right or left swipe gesture may beused to access and/or present the visual information for the differentfiles under the particular parent folder or directory. The presentedvisual information may include the name of the folder, directory, orfile, or may include an icon associated with the folder, directory, orfile.

The user may access a particular folder by stopping rotation of 3D GUIelement 700 when the visual information for the particular folder ispresented in 3D GUI element 700. In some embodiments, the differentvisual information or render positions within the point cloud may beassociated or linked to different folders and/or files. Accordingly,second user input may be issued with respect to 3D GUI element 700 toaccess (at 706) the particular folder. For instance, the second userinput may include a double tap gesture when the visual information forthe particular folder is displayed within 3D GUI element 700.

In response to the second user input being issued when the visualinformation for the particular folder is displayed within 3D GUI element700, the 3D GUI may open or access the particular folder, and/or mayadjust 3D GUI element 700 to present visual information for the files orfolders that are stored under the particular folder.

Other inputs or gestures may be associated with other file systemcommands. For instance, a two finger expand gesture performed over thevisual information for a specific folder may invoke a copy command thatcopies the specific folder and its contents. Single finger swipegestures and double tap gestures may then be used to navigate to adifferent location in the file system, and a two finger pinch gesturemay paste the copied specific folder and its contents to the navigatedto location within the file system. In some embodiments, a two fingertap gesture performed over the visual information for a specific folderor file may cause the 3D GUI or file system to delete that folder orfile.

FIG. 8 presents a process 800 for presenting and interacting with a 3DGUI element in accordance with some embodiments presented herein.Process 800 may be implemented by the 3D GUI that runs on hardwareresources of a user device. The 3D GUI may be implemented within thedevice operating system (“OS”), within different OS services (e.g., afile system), and/or within different applications running atop the OS.

Process 800 may include receiving (at 802) a point cloud or other 3Dimage file. The point cloud may include data points that are positionedin 3D space based on positional elements defined for each data point,and that are rendered with different visual characteristics based ondifferent non-positional elements defined for each data point. Thecollective rendering of the data points from a single point cloud or 3Dimage file may generate a 3D icon, 3D emoji, 3D wallpaper, and/or other3D GUI element that includes different visual information at differentpositions within the 3D space represented by that 3D GUI element. Insome embodiments, the 3D GUI may dynamically generate the point cloudbased on data associated with applications, functions, folderstructures, and/or 2D GUI elements that are to be accessed from the 3DGUI element represented by the point cloud or other 3D image file.

Process 800 may include determining (at 804) different sets of the datapoints within the point cloud that render and/or provide the visualinformation for identifying different objects, data, commands,functions, API calls, folders, files, and/or 2D GUI elements that areaccessible from the 3D GUI element that is generated based on the pointcloud or 3D image file data points. In some embodiments, the differentsets of data points may be tagged to identify what they represent. Forinstance, a first set of data points within a point cloud may include anon-positional element with a “Browser Back” value, a second set of datapoints within the point cloud may include the value “Browser Forward”for the non-positional element, and a third set of data points withinthe point cloud may include the value “Browser Halt” for thenon-positional element. In some embodiments, the 3D GUI may render thepoint cloud, and perform image or pattern recognition to tag thedifferent sets of data points with different identifiers for thecommand, function, API call, folder, file, and/or 2D GUI elementrepresented by that set of data points.

Process 800 may include defining (at 806) rules that control whatcommands, functions, API calls, scripts, executable code, and/or otheractions are performed or invoked in response to different inputs beingperformed on or to the part of the 3D GUI element that is represented byeach set of data points. In some embodiments, the point cloud or 3Dimage file may be associated with a particular application that runs onthe user device, and the particular application may include an inputdefinition that defines the rules for the application commands,functions, API calls, scripts, executable code, and/or other actionsthat are performed or invoked in response to the different input beingapplied to the visual information associated with the different sets ofdata points. For instance, defining (at 806) the rules may includedetermining a first application function to perform or call with zero ormore variables when a particular swipe gesture is performed on or over afirst data point set (e.g., a first set of data points) of the pointcloud tagged with the “Browser Back” value, and determining a secondapplication function to perform or call with zero or more variables whenthe particular swipe gesture is performed on or over a second data pointset of the point cloud tagged with the “Browser Forward” value. Theinputs may include touch inputs that are detected by a touch sensor on atrackpad, touchscreen, or other touch-sensitive input device. The inputsmay also include mouse, keyboard, voice, and/or means with which a usermay interact with and/or control the device as well as the applicationfeatures and functionality. The invocable commands, functions, scripts,API calls, executable code, and/or other actions may be specified to theapplication that is accessed via the 3D GUI element represented by thepoint cloud, OS functions, system calls, and/or other non-applicationspecific functionality supported by the application or device. Theinvocable output may also include rotating, manipulating, or otherwiseadjusting the rendering of the point cloud to present different visualinformation therein or to generate various 3D effects by manipulatingthe positional and/or non-positional elements of the existing datapoints in response to certain inputs or no inputs.

Process 800 may include rendering (at 808) a 3D GUI element from thepositional and non-positional elements of the point cloud or 3D imagefile data points, and presenting (at 810) the 3D GUI element within a 3DGUI. In some embodiments, the point cloud, 3D image file, and/or 3D GUImay define a default or starting position from which to render the datapoints. In some embodiments, the 3D GUI may include a GUI that is usedto launch or access different functionality of different applicationsconfigured on the device. In some embodiments, the 3D GUI may include aspecific GUI of an application that becomes accessible when theapplication is launched, active, open, and/or frontmost.

Process 800 may include manipulating (at 810) the 3D GUI element basedon any static rules that are defined for the 3D GUI element and that areinvoked independent of user input. For instance, manipulating (at 810)the 3D GUI element may include continuously rendering the point cloudfrom different render positions to simulate a 3D rotation or movement ofthe 3D GUI element.

Process 800 may include receiving (at 812) user input that is providedin association with the 3D GUI element. For instance, the user input maybe provided when the 3D GUI element is selected, or may be provided overor on a part of the 3D GUI element.

Process 800 may include determining (at 814) the particular set of datapoints within the 3D GUI element that are targets of the user input. Theparticular set of data points may include the set of data points thatare centered or frontmost on the 3D GUI element when the user input isreceived (at 812), or may include the set of data points that are at aposition where the user input begins and/or ends.

Process 800 may include executing (at 816) the action from the rule thatis defined for the particular set of data points and the user input. Insome embodiments, executing (at 816) the action may include executing acommand, function call, API call, script, and/or executable code of theassociated application. The rule may further specify one or morevariables for the executed action based on the particular set of datapoints. For instance, the particular set of data points may correspondto a visualization of a first email, and executing (at 816) the actionmay include invoking an application call that accesses, opens, and/orotherwise implicates the first email. In some embodiments, executing (at816) the action may include adjusting the visualization of the 3D GUIelement by rendering the point cloud from different render positionsthat center on and/or bring different sets of data points to focus orthe center, and/or by changing the positional and/or non-positionalelements of different data points to create various 3D effects withinthe 3D GUI element. Accordingly, the 3D GUI may modify the data pointsof the point cloud, and may store the modified point cloud to retain thecurrent state of the 3D GUI element.

FIG. 9 is a diagram of example components of device 900. Device 900 maybe used to implement one or more of the tools, devices, or systemsdescribed above (e.g., the 3D GUI 201, the user device that generatesand presents the 3D icons, 3D emojis, 3D wallpapers, and/or 3D GUIelements, etc.). Device 900 may include bus 910, processor 920, memory930, input component 940, output component 950, and communicationinterface 960. In another implementation, device 900 may includeadditional, fewer, different, or differently arranged components.

Bus 910 may include one or more communication paths that permitcommunication among the components of device 900. Processor 920 mayinclude a processor, microprocessor, or processing logic that mayinterpret and execute instructions. Memory 930 may include any type ofdynamic storage device that may store information and instructions forexecution by processor 920, and/or any type of non-volatile storagedevice that may store information for use by processor 920.

Input component 940 may include a mechanism that permits an operator toinput information to device 900, such as a keyboard, a keypad, a button,a switch, etc. Output component 950 may include a mechanism that outputsinformation to the operator, such as a display, a speaker, one or moreLEDs, etc.

Communication interface 960 may include any transceiver-like mechanismthat enables device 900 to communicate with other devices and/orsystems. For example, communication interface 960 may include anEthernet interface, an optical interface, a coaxial interface, or thelike. Communication interface 960 may include a wireless communicationdevice, such as an infrared (“IR”) receiver, a Bluetooth® radio, or thelike. The wireless communication device may be coupled to an externaldevice, such as a remote control, a wireless keyboard, a mobiletelephone, etc. In some embodiments, device 900 may include more thanone communication interface 960. For instance, device 900 may include anoptical interface and an Ethernet interface.

Device 900 may perform certain operations relating to one or moreprocesses described above. Device 900 may perform these operations inresponse to processor 920 executing software instructions stored in acomputer-readable medium, such as memory 930. A computer-readable mediummay be defined as a non-transitory memory device. A memory device mayinclude space within a single physical memory device or spread acrossmultiple physical memory devices. The software instructions may be readinto memory 930 from another computer-readable medium or from anotherdevice. The software instructions stored in memory 930 may causeprocessor 920 to perform processes described herein. Alternatively,hardwired circuitry may be used in place of or in combination withsoftware instructions to implement processes described herein. Thus,implementations described herein are not limited to any specificcombination of hardware circuitry and software.

The foregoing description of implementations provides illustration anddescription, but is not intended to be exhaustive or to limit thepossible implementations to the precise form disclosed. Modificationsand variations are possible in light of the above disclosure or may beacquired from practice of the implementations.

The actual software code or specialized control hardware used toimplement an embodiment is not limiting of the embodiment. Thus, theoperation and behavior of the embodiment has been described withoutreference to the specific software code, it being understood thatsoftware and control hardware may be designed based on the descriptionherein.

For example, while series of messages, blocks, and/or signals have beendescribed with regard to some of the above figures, the order of themessages, blocks, and/or signals may be modified in otherimplementations. Further, non-dependent blocks and/or signals may beperformed in parallel. Additionally, while the figures have beendescribed in the context of particular devices performing particularacts, in practice, one or more other devices may perform some or all ofthese acts in lieu of, or in addition to, the above-mentioned devices.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of the possible implementations. Infact, many of these features may be combined in ways not specificallyrecited in the claims and/or disclosed in the specification. Althougheach dependent claim listed below may directly depend on only one otherclaim, the disclosure of the possible implementations includes eachdependent claim in combination with every other claim in the claim set.

Further, while certain connections or devices are shown, in practice,additional, fewer, or different, connections or devices may be used.Furthermore, while various devices and networks are shown separately, inpractice, the functionality of multiple devices may be performed by asingle device, or the functionality of one device may be performed bymultiple devices. Further, while some devices are shown as communicatingwith a network, some such devices may be incorporated, in whole or inpart, as a part of the network.

To the extent the aforementioned embodiments collect, store or employpersonal information provided by individuals, it should be understoodthat such information shall be used in accordance with all applicablelaws concerning protection of personal information. Additionally, thecollection, storage and use of such information may be subject toconsent of the individual to such activity, for example, throughwell-known “opt-in” or “opt-out” processes as may be appropriate for thesituation and type of information. Storage and use of personalinformation may be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

Some implementations described herein may be described in conjunctionwith thresholds. The term “greater than” (or similar terms), as usedherein to describe a relationship of a value to a threshold, may be usedinterchangeably with the term “greater than or equal to” (or similarterms). Similarly, the term “less than” (or similar terms), as usedherein to describe a relationship of a value to a threshold, may be usedinterchangeably with the term “less than or equal to” (or similarterms). As used herein, “exceeding” a threshold (or similar terms) maybe used interchangeably with “being greater than a threshold,” “beinggreater than or equal to a threshold,” “being less than a threshold,”“being less than or equal to a threshold,” or other similar terms,depending on the context in which the threshold is used.

No element, act, or instruction used in the present application shouldbe construed as critical or essential unless explicitly described assuch. An instance of the use of the term “and,” as used herein, does notnecessarily preclude the interpretation that the phrase “and/or” wasintended in that instance. Similarly, an instance of the use of the term“or,” as used herein, does not necessarily preclude the interpretationthat the phrase “and/or” was intended in that instance. Also, as usedherein, the article “a” is intended to include one or more items, andmay be used interchangeably with the phrase “one or more.” Where onlyone item is intended, the terms “one,” “single,” “only,” or similarlanguage is used. Further, the phrase “based on” is intended to mean“based, at least in part, on” unless explicitly stated otherwise.

1. A method comprising: receiving a plurality of three-dimensional(“3D”) images, each 3D image of the plurality of 3D images comprising aplurality of image data that is distributed across a 3D space of that 3Dimage; defining a first render position in the 3D space of each 3D imageof the plurality of 3D images; identifying a different first set of theplurality of image data from each 3D image of the plurality of 3D imagesthat becomes visible from the first render position in each 3D image;generating a plurality of 3D graphical user interface (“GUI”) elementsin a GUI by rendering the different first set of image data that becomesvisible from the first render position defined in each 3D image, whereinthe plurality of 3D GUI elements in the GUI are separate from theplurality of 3D images and present a smaller subset of the 3D spacespanned by the plurality of 3D images; detecting an input that isapplied to a particular 3D GUI element of the plurality of 3D GUIelements; changing from the first render position to a different secondrender position in the 3D space of a particular 3D image of theplurality of 3D images in response to the input that is applied to theparticular 3D GUI element, wherein the particular 3D GUI element isgenerated from the different first set of image data of the particular3D image; identifying a different second set of the plurality of imagedata from the particular 3D image that becomes visible from the secondrender position in the particular 3D image; and changing the particular3D GUI element in the GUI by rendering the different second set of theplurality of image data from the second render position of theparticular 3D image.
 2. The method of claim 1 further comprising:updating the GUI by presenting the particular 3D GUI element with therendering of the different second set of image data from the secondrender position in the 3D space of the particular 3D image, and bypresenting other GUI elements from the plurality of 3D GUI elements withthe rendering of the different first set of image data from the firstrender position in the 3D space of other 3D images of the plurality of3D images that are not the particular 3D image.
 3. The method of claim 1further comprising: presenting each 3D GUI element of the plurality of3D GUI elements as a different volumetric object in the GUI; and whereinchanging the particular 3D GUI element comprises rotating a particularvolumetric object of the particular 3D GUI element by renderingdifferent image data from the plurality of image data in the particular3D image that becomes visible between the first render position and thesecond render position.
 4. The method of claim 1, wherein changing theparticular 3D GUI element comprises: animating the particular 3D GUIelement by presenting image data from the plurality of image data of theparticular 3D image that is between the different first set of imagedata of the particular 3D image and the different second set of imagedata of the particular 3D image.
 5. The method of claim 1, wherein theinput is a first user input, the method further comprising: mapping afirst executable action to the different first set of image data of theparticular 3D image, and a second executable action to the differentsecond set of image data of the particular 3D GUI element; performingthe first executable action in response to a second user input beingissued while rendering the different first set of image data for theparticular 3D GUI element; and performing the second executable actionin response to the second user input being issued while rendering thedifferent second set of image data for the particular 3D GUI element. 6.The method of claim 1 further comprising: mapping an executable actionof a first application to the different first set of image data of theparticular 3D image; and mapping an executable action of a secondapplication to the different first set of image data of a second 3Dimage of the plurality of 3D images that is different than theparticular 3D image and that is used to generate a second 3D GUI elementof the plurality of GUI elements, wherein the particular 3D GUI elementis a 3D icon for accessing functionality of the first application, andthe second 3D GUI element is a 3D icon for accessing functionality ofthe second application.
 7. The method of claim 1 further comprising:detecting a user interaction with a second 3D GUI element of theplurality of 3D GUI elements that is different than the particular 3DGUI element; determining an updated render position in the 3D space of asecond 3D image of the plurality of 3D images that is associated withthe second 3D GUI element based on the user interaction with the second3D GUI element; and changing of the second 3D GUI element in the GUI byrendering new image data that becomes visible from the updated renderposition in the 3D space of the second 3D image.
 8. The method of claim1, wherein the input is a user input for rotating the particular 3D GUIelement in a particular direction.
 9. The method of claim 1, whereindetecting the input comprises receiving a notification from a particularapplication that is accessed using the particular 3D GUI element; andwherein rendering the different second set of image data comprisespresenting the notification from the particular application within theparticular 3D GUI element.
 10. The method of claim 1, wherein detectingthe input comprises detecting a new message in an email, messaging, orsocial media application that is accessed using the particular 3D GUIelement; and wherein rendering the different second set of image datacomprises determining that the second set of image data comprises animage of a sender of the new message and presenting the image of thesender in the particular 3D GUI element in response to changing thevisualization of the particular 3D GUI element.
 11. The method of claim1 further comprising: determining a rule that is defined for the inputand the particular 3D GUI element, wherein the rule links an executableaction to the different second set of image data; and performing theexecutable action in response to detecting the input being applied tothe particular 3D GUI element and the particular 3D GUI elementpresenting a visualization that is generated from rendering thedifferent second set of image data.
 12. The method of claim 1, whereinrendering the different first set of image data comprises presenting thedifferent first set of image data from the particular 3D image in acenter or a foreground of the particular 3D GUI element, and presentingthe different second set of image data from the particular 3D imagebehind or to a side of the different first set of image data in theparticular 3D GUI element; and wherein rendering the different secondset of image data comprises presenting the different second set of imagedata from the particular 3D image in the center or the foreground of theparticular 3D GUI element, and presenting the different first set ofimage data from the particular 3D image behind or to a side of thedifferent second set of image data in the particular 3D GUI element. 13.(canceled)
 14. The method of claim 1, wherein the particular 3D image isa point cloud comprising a plurality of data points distributed acrossthe 3D space represented by the particular 3D image, the different firstset of image data from the particular 3D image comprises a first set ofthe plurality of data points distributed in a first region of the 3Dspace, and the different second set of image data from the particular 3Dimage comprises a second set of the plurality of data points distributedin a second region of the 3D space.
 15. A device comprising: one or moreprocessors configured to: receive a plurality of three-dimensional(“3D”) images, each 3D image of the plurality of 3D images comprising aplurality of image data that is distributed across a 3D space of that 3Dimage; define a first render position in the 3D space of each 3D imageof the plurality of 3D images; identify a different first set of theplurality of image data from each 3D image of the plurality of 3D imagesthat becomes visible from the first render position in each 3D image;generate a plurality of 3D graphical user interface (“GUI”) elements ina GUI by rendering the different first set of image data that thatbecomes visible from the first render position defined in each 3D image,wherein the plurality of 3D GUI elements in the GUI are separate fromthe plurality of 3D images and present a smaller subset of the 3D spacespanned by the plurality of 3D images; detect an input that is appliedto a particular 3D GUI element of the plurality of 3D GUI elements;change from the first render position to a different second renderposition in the 3D space of a particular 3D image of the plurality of 3Dimages in response to the input that is applied to the particular 3D GUIelement, wherein the particular 3D GUI element is generated from thedifferent first set of image data of the particular 3D image; identify adifferent second set of the plurality of image data from the particular3D image that becomes visible from the second render position in theparticular 3D image; and change the particular 3D GUI element in the GUIby rendering the different second set of the plurality of image datafrom the second render position of the particular 3D image.
 16. Thedevice of claim 15, wherein the one or more processors are furtherconfigured to: update the GUI by presenting the particular 3D GUIelement with the rendering of the different second set of image datafrom the second render position in the 3D space of the particular 3Dimage, and by presenting other GUI elements from the plurality of 3D GUIelements with the rendering of the different first set of image datafrom the first render position in the 3D space of other 3D images of theplurality of 3D images that are not the particular 3D image.
 17. Thedevice of claim 15, wherein the one or more processors are furtherconfigured to: present each 3D GUI element of the plurality of 3D GUIelements as a different volumetric object in the GUI; and whereinchanging the particular 3D GUI element comprises rotating a particularvolumetric object of the particular 3D GUI element by renderingdifferent image data from the plurality of image data in the particular3D image that becomes visible between the first render position and thesecond render position.
 18. The device of claim 15, wherein changing theparticular 3D GUI element comprises: animating the particular 3D GUIelement by presenting image data from the plurality of image data of theparticular 3D image that is between the different first set of imagedata of the particular 3D image and the different second set of imagedata of the particular 3D image.
 19. The device of claim 15, whereinchanging the visualization of the particular 3D GUI element comprises:detect a user interaction with a second 3D GUI element of the pluralityof 3D GUI elements that is different than the particular 3D GUI element;determine an updated render position in the 3D space of a second 3Dimage of the plurality of 3D images that is associated with the second3D GUI element based on the user interaction with the second 3D GUIelement; and change the second 3D GUI element in the GUI by renderingnew image data that becomes visible from the updated render position inthe 3D space of the second 3D image.
 20. A non-transitorycomputer-readable medium storing instructions that, when executed by oneor more processors of a three-dimensional (“3D”) interface, cause the 3Dinterface to perform operations comprising: receive a plurality of 3Dimages, each 3D image of the plurality of 3D images comprising aplurality of image data that is distributed across a 3D space of that 3Dimage; define a first render position in the 3D space of each 3D imageof the plurality of 3D images; generate a plurality of 3D graphical userinterface (“GUI”) elements by rendering the different first set of imagedata that becomes visible from the first render position defined in each3D image, wherein the plurality of 3D GUI elements in the GUI areseparate from the plurality of 3D images and present a smaller subset ofthe 3D space spanned by the plurality of 3D images; detect an input thatis applied to a particular 3D GUI element of the plurality of 3D GUIelements; change from the first render position to a different secondrender position in the 3D space of a particular 3D image of theplurality of 3D images in response to the input that is applied to theparticular 3D GUI element, wherein the particular 3D GUI element isgenerated from the different first set of image data of the particular3D image; identify a different second set of the plurality of image datafrom the particular 3D image that becomes visible from the second renderposition in the particular 3D image; and change the particular 3D GUIelement in the GUI by rendering the different second set of theplurality of image data from the second render position of theparticular 3D image.